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Reports on the condition of STEM learning in the U.S. 


LEFT TO CHANCE: 

U.S. Middle Schoolers Lack in-Depth Experience 
With Technology and Engineering 


Technology and engineering have played central roles 
in forming our national identity. We see the United 
States as a nation of tinkerers and inventors who have 
helped chart the course of global innovation for 
centuries. Only time will tell if we can continue to live 
up to this conception of ourselves. As technology 
and engineering come to affect almost every aspect 
of modern life, this is no small matter. 

New data from the first-ever Nation's Report Card in 
Technology and Engineering Literacy (TEL) are not reas- 
suring. Well less than half the nation's eighth-graders are 
on track to become proficient in a set of skills they will 
need to thrive in society and the workplace. Low-income 
and minority youth lag farthest behind. 1 

On closer examination, these findings should not sur- 
prise us. Change the Equation's analysis of survey data from 
TEL reveals that millions of American youth spend precious 
little time tinkering, troubleshooting, or doing the kinds of 
hands-on problem-solving that are at the heart of technol- 
ogy and engineering. Girls, minorities, and low-income 
students do least of all — dampening hopes to create a 
more diverse STEM workforce in future years. Without 


Analyzing the TEL survey 

This brief presents Change the Equation's 
analysis of results from survey questions TEL test 
takers answered after they completed the TEL 
assessment. A nationally representative sample 
of 21,500 eighth-graders in 840 U.S. public and 
private schools completed the survey, answering 
questions about how often they have learned 
about technology and engineering or taken part 
in technology and engineering activities, both in 
and out of school. It is beyond the scope of this 
brief to provide full survey results. 

For more information about the TEL survey 
methodology, see nationsreportcard.gov. 

intentional strategies to expose many more young people 
to technology and engineering, we are leaving a critical 
aspect of students' education to chance. 


Millions of American youth spend precious little time tinkering, troubleshooting, or doing 
the kinds of hands-on problem-solving at the heart of technology and engineering. 
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Technology and engineering "literacy": It's not 
just for techies and engineers anymore 
Of course, not everyone has to become an engineer or 
technology professional — though both are certainly in 
demand . 2 Rather, the TEL assessment gauges eighth- 
graders' technology and engineering literacy, focusing 
on the "knowledge and competencies ... all students 
and citizens [need ]." 3 

TEL's expansive definitions of technology and engineer- 
ing support this focus on universal skills. Technology is 
"any modification of the natural world done to fulfill human 
needs or desires," which encompasses anything "from 
paper to the internet." Engineering is "a systematic and 
often iterative approach to designing objects, processes, 
and systems to meet human needs and wants." People 
literate in these fields might not be able to design a bridge 
or program in Java, but they "can apply their understanding 
of technology principles to real-life situations ." 4 

Master technology, or be mastered by it 
Those who are literate in these fields can better under- 
stand and benefit from the major engines of economic 
growth 5 by taking full advantage of technology to address 
challenges in their jobs, their communities, and their lives. 
People who are not literate in engineering and technology 
are too often doomed to be replaced by the technologies 
they cannot command. 


Those who understand technology and engineering 
are also best equipped to advocate for them as forces that 
promote the common good. At a time when even some of 
the world's most respected innovators worry that technol- 
ogy could bring as much peril as promise , 6 the nation needs 
citizens who can ensure that technology does not stray too 
far from its fundamentally humane purpose. 

A vision of technology and engineering literacy 
embraced by employers 

TEL's vision of literacy in technology and engineering dove- 
tails with skills employers value, such as the ability to apply 
knowledge to new situations, to identify and solve unex- 
pected problems without a playbook, or to learn through 
ingenuity, failure and perseverance . 7 

The TEL survey gauges how much exposure eighth- 
graders have to environments and experiences, both in 
and out of school, that can promote this vision of learning. 
Do they ever have the opportunity to brainstorm designs 
in an iterative fashion to address a specific challenge, for 
example? Do they ever have to design something with 
limited time or resources? Do they often take things apart 
to see how to fix them? Do they build models to solve a 
problem? Do they create or edit digital media? 


54 %* 



classes 


* Statistically significant difference from higher income 
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Figure 2 

Percent of eighth-graders who report taking "none" 



White White Black Hispanic Black Hispanic 

male female male male female female 


* Statistically significant difference from white male 


Most eighth-graders have encountered some 
engineering or technology in school 
At first blush, the data on students' overall exposure to 
technology or engineering do not seem dire, even if they 
provide little cause for celebration. Asked if they have 
"ever" studied engineering or technology concepts in 
their math, science, history, or social studies classes, 

76 percent say yes. Statistically significant gaps separate 
lower-income 8 students (74 percent) from their higher- 
income peers (78 percent), but strong majorities of 
students from each race, income level, and gender say 
they have at least encountered the topics in school. 


That, of course, is a low bar. Many fewer eighth- 
graders — roughly half (52 percent) — say they have ever 
taken a course that focuses on engineering or technology. 
Here, the gaps between lower- and higher-income stu- 
dents are stark. (See figure 1) 

Girls of color face the worst odds of all. For example, 
more than three out of five Hispanic girls say they have 
never taken any of these courses. (See figure 2) 

Yet course titles in and of themselves tell us relatively 
little about the actual experiences students have in school. 
Other TEL survey findings suggest that, if anything, these 
data on course participation overstate students' engage- 
ment in hands-on technology and engineering. 


People who are not literate in engineering and technology are too often doomed 
to be replaced by the technologies they cannot command. 
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Eighth-graders have limited opportunities to 
tinker or troubleshoot in school 
A mere 9 percent of eighth-graders attend schools that 
report placing "a lot" of emphasis on "figuring out how 
to fix something," and another 37 percent attend schools 
that tepidly report "some" emphasis. A full 49 percent 
attend schools that report little or no emphasis. Schools 
provide a very similar account of their emphasis on "finding 
the right people to work with or get help from to fix some- 
thing." (See figure 3) 

Most eighth-graders report that, in school, they seldom 
actually try their hand at fixing things. Only 42 percent say 
that, at least three times in their entire school careers, they 
have "figured out why something is not working in order 
to fix it." Just 26 percent report that, at least three times, 
they have "taken something apart in order to fix it or see 
how it works." A full 43 percent say they have never done 
so. (See figure 4) 

Eighth-graders are moderately more likely to say they 
select and use tools or build models in school to solve 
problems, but here, too, such experiences seem few and far 
between for most students. (See figure 4) 

Such activities are staples of how an engineer approaches 
real-world challenges, and they can also have a strong 
impact in schools. Decades of research suggest that people 


Figure 3 

In your school, prior to or in eighth grade, how 
much emphasis is placed on teaching students: 
(school reported) 

To figure out how To find the right people 

to fix something to work with/get help 

from to fix something 



■ A lot ■ Some A little None ■ Don't know 

often learn best by testing solutions to real-world problems 
through hands-on trial and error. 9 If the TEL survey results 
are any indication, this research has had little impact on the 
nation's schools. 


Figure 4 

In school, how often have you ever done the following activities? 


Figured out why 
something is not 
working in order 
to fix it 


Taken something 
apart in order to 
fix it or see how it 
works 


Used different tools, 
machines, or materials 
to see which are best 
for the given purpose 


Built or tested a 
model to see if it 
solves a problem 



■ Six-plus times ■ Three/five times Once/Twice Never 
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Eighth-graders are more likely to tinker and 
troubleshoot outside of school than in school 

Facing limited opportunities in their schools to roll up 
their sleeves, take things apart, and explore how they 
work, students say they are more likely to do so when 
school is out. (See figure 5) 

Even when tinkering or troubleshooting activities are 
sophisticated enough to require tools, machines or models, 


students remain somewhat more likely to do them outside 
of schools than in them. (See figure 5) 

In absolute terms, these percentages are far from heart- 
ening. If we are to believe their own account, a large share of 
the nation's eighth graders has engaged in such activities no 
more than two times in their lives, whether in or out of school 


Figure 5 

How often have you ever done the following activities? 


Figured out why 
something is not 
working in order 
to fix it 



school school 


Taken something 
apart in order to 
fix it or see how 
it works 



Out of In 

school school 


Used different tools, 
machines, or materials 
to see which are best 
for the given purpose 



Out of In 

school school 


Built or tested a 
model to see if it 
solves a problem 



Out of In 

school school 


■ Six-plus times ■ Three/five times 


* Statistically significant difference from in school 


Gaps in opportunity are larger outside 
of schools than in them 

More eighth-graders engage in hands-on engineering 
outside of school than in school, but those eighth-graders 
are a much less diverse group. Figure 6 on the following 
page, which lays out the percentages of students who 
have participated in such activities at least six times, 
illustrates large inequities of income, race, and gender. 
The gray and red cells highlight gaps that are statisti- 
cally significant. Each gray cell indicates an area in which 


minority students lag behind white students, lower-income 
students lag behind higher-income students, or girls lag 
behind boys. Each red cell, by contrast, indicates a statis- 
tically significant advantage for minority, lower-income, or 
female students over their white, higher-income, or male 
peers. Any disadvantages minorities, girls, and lower- 
income students face in school pale in comparison to what 
they face outside of school. 

Not only are there more gaps outside of school, those 
out-of-school gaps are larger. 10 If anything, schools are 
softening the disadvantage low-income students, minorities. 
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Figure 6 

Percentage of students who say they have done the following six or more times 


TAKEN SOMETHING APART TO FIX IT OR SEE HOW IT WORKS 


Income Race/Ethnicity Gender 


In school 

Higher 

Lower 

White 

Black 

Latino 

11% 

Male 

16% 

Female 

11% 

13% 

11% 

15% 

8% 

Out of school 

32% 

28% 

33% 

26% 

25% 

39% 

21% 

FIGURED OUT WHY SOMETHING IS NOT WORKING IN ORDER TO FIX IT 



Income 


Race/Ethnicit 

y 

Gender 


Higher 

Lower 

White 

Black 

Latino 

Male 

Female 

In school 

19% 

17% 

18% 

20% 

16% 

21% 

16% 

Out of school 

41% 

33% 

42% 

31% 

30% 

43% 

31% 


BUILT OR TESTED A MODEL TO SOLVE A PROBLEM 


Income Race/Ethnicity Gender 



Higher 

Lower 

White 

Black 

Latino 

Male 

Female 

In school 

15% 

12% 

14% 

14% 

11% 

14% 

13% 

Out of school 

19% 

16% 

20% 

14% 

13% 

23% 

_ 

12% j 


USED DIFFERENT TOOLS, MATERIALS, OR MACHINES TO SEE WHICH ARE BEST FOR A GIVEN PURPOSE 


Income Race/Ethnicity Gender 



Higher 

Lower 

White 

Black 

Latino 

Male 

Female 

In school 

18% 

15% 

18% 

16% 

13% 

18% 

15% 

Out of school 

28% 

21% 

29% 

18% 

19% 

29% 

20% 


and girls face in some areas — if only slightly. Because 
so few eighth-graders frequently take part in school 
hands-on activities that promote engineering literacy, how- 
ever, those mitigating effects can have an anemic impact 
at best. 

Outside of school, girls of color fare especially badly, 
suffering disadvantage heaped upon disadvantage. To cite 
one finding not displayed in the table above, 44 percent 
of white boys, but only 19 percent of Hispanic girls, have 
taken something apart out of school more than five times 
in their lives. 


Digital media are ubiquitous, but students 
consume much more than they create 
Eighth-graders report that they do not often use digital 
technology to create spreadsheets, presentations, or 
digital media, either in or out of schools. At best, only 
about half of students have done so more than a few 
times a year in school. The data look even worse out of 
school. (See figure 7) 

The relative rarity of spreadsheets and presentations 
in and out of school may not come as a surprise, even 


Any disadvantages minorities, girls, and lower-income students face in school pale 
in comparison to what they face outside of school. 
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Outside of schools, students' exposure to engineering and technology is left 
largely to chance. 


though they are staples of the work world. Today's 
technology advocates may be more disappointed by 
the finding that students spend little time creating, 
editing, and organizing digital media. This finding flies 
in the face of a hope technology advocates have long 
held: namely, that digital technology would prompt 
many more students to create, rather than mostly con- 
sume, innovative digital products. 

Other recent research bears out this finding and offers 
more insights into how young people use digital technol- 
ogy. A robust 2015 survey of U.S. tweens by Common 
Sense Media found that, of all the time they spent out 
of school using computers, tablets, and smartphones, 
tweens devoted only three percent to creating content. In 
fact, creating digital art, writing, making videos, creating/ 
modifying games, coding, and creating digital music all 
fell to the very bottom of tweens' list of "favorite media 
activities," far below activities like playing games, using 
social media, watching videos, and listening to music. 
None of these creative activities earned the top spot for 
more than two percent of tweens, and most did not even 
reach one percent. 11 

Outside of schools, persistent "digital divides" continue 
to limit some young people's access to the digital tools 
they need to create or edit content. The TEL survey does 
not shed light on students' access to digital technology at 


home, but other recent research reveals that low-income 
and minority students are much less likely than their peers 
to have high speed internet and uninterrupted access to 
digital devices. 12 Teachers of lower-income students are 
much more likely to say these barriers inhibit teachers' abil- 
ity to integrate technology into their classrooms and assign 
homework that incorporates technology in a meaningful 
way. 13 Future TEL surveys could explore these issues in 
greater depth. 

Many students' exposure to technology and 
engineering is left largely to chance 
The fact that out-of-school inequities match or often 
even dwarf those in schools should come as little surprise. 
Outside of schools, students' exposure to engineering 
and technology is left largely to chance. Those without 
family members in fields like engineering or the trades, for 
example, may have fewer opportunities to learn how to fix 
things and explore how they work. Those who lack access 
to computers, other digital devices, or high speed internet 
are in the worst position to teach themselves about digital 
media. 

The vast majority of eighth-graders turn to their family, 
their friends, or even themselves to learn how things work 
or how to collect and share digital media. Well more than 
half say family members have taught them most about 


Figure 7 

How often do you use a computer or other digital technology for the following activities? 



CREATE 

SPREADSHEETS 

CREATE 

PRESENTATIONS 

CREATE/EDIT/ 
ORGANIZE DIGITAL MEDIA 


Never/ 
almost never 

Few times/ 
year 

Never/ 
almost never 

Few times/ 
year 

Never/ 
almost never 

Few times/ 
year 

In school* 

30% 

33% 

10% 

37% 

24% 

26% 

Out of school* 

57% 

22% 

45% 

27% 

38% 

21% 


* Questions read "for school work" and "not related to your schoolwork." 
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Figure 8 

Who taught you most about building things, fixing things, or how things work? 


19% 

63% 


13% 




At least 85% say others taught them more than their teachers did 



■ I taught myself ■ Family members ■ Friends Teachers Someone else 


Figure 9 

Who taught you most about using computers or other digital 

technology for collecting and sharing information? 7% 2% 


37% 

30% 


25% 




At least 74% say others taught them more than their teachers did 

■ I taught myself ■ Family members ■ Friends Teachers Someone else 


"building things, fixing things, or how things work." The 
remaining students are more likely to credit themselves 
(1 9 percent) than their teachers (1 3 percent) for teaching 
them these engineering skills. (See figure 8) 

Eighth-graders are even more likely to consider them- 
selves essentially self-taught in technology. A plurality, 37 
percent, say they taught themselves most of what they 
know about "using computers or other digital technology 
for collecting and sharing information." Teachers, by con- 
trast, were most influential for only 25 percent of students. 
(See figure 9) 

By eighth grade, most students' experience with 
engineering and technology activities has been an accident 
of birth and circumstance. 


TEL's critical lessons for engaging more girls in 
technology and engineering 

TEL is an object lesson in how to encourage girls to pursue 
technology and engineering. 

One of the TEL assessment's most notable findings is that 
girls outscored boys by a small but statistically-significant 
margin. Yet herein lies a paradox. Girls are less likely to say 
they have engaged in many of the important hands-on activi- 
ties addressed in the TEL survey: figuring out why something 
is not working, taking things apart, using tools, and building 
models to solve problems. Why the apparent disconnect? 

Part of the answer lies in the portion of the test where 
girls enjoyed the largest advantage over boys: the test's 
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Educators who harness TEL's vision of literacy in technology and engineering may 
well attract many more girls to those fields. 


Information and Communication Technologies (ICT) 
subscale, which focuses on "accessing, creating, and 
communicating information and ... facilitating creative 
expression ." 14 Girls have outperformed boys in other tests 
of ICT skills, possibly because girls tend to perform better 
in reading and communication . 15 Yet even in other areas of 
the assessment, girls scored on par with boys — no great 
accomplishment in absolute terms, but better than their 
lagging participation in hands-on activities would suggest. 

This apparent paradox also conceals a larger lesson 
about the strength of the TEL framework and what we can 
learn from it. TEL plays to girls' strengths by situating its 
tasks in narratives about improving communities or helping 
people. These are precisely the contexts that, according 
to decades of research, are most likely to interest girls in 
subjects like technology and engineering . 16 

Educators who harness TEL's vision of literacy in technol- 
ogy and engineering may well attract many more girls to 
those fields. The TEL results are an important reminder that 
we are squandering much of the nation's female talent. If 
we could give more girls compelling reasons to enter their 
garages, workshops, and makerspaces to explore solutions 
to real-world problems through hands-on engineering activ- 
ities, the nation would be much richer for it. 

Concrete strategies for ensuring widespread 
literacy in technology and engineering 
We cannot simply leave young peoples' technology and 
engineering literacy to chance. The challenges we face 
require strong, coordinated efforts among government, 
education, and community leaders at the federal, state, 
local levels. 

Fortunately, conditions are ripe for such efforts. A grow- 
ing number of states have been integrating technology 
and engineering literacy into their vision for what every 
student should learn. Still, it would be far too easy to let 
this moment slip by. State and local priorities can change 
quickly in today's volatile political environment. Advocates 
need to keep state and local leaders' attention firmly on 
concrete and practical strategies for promoting and attain- 
ing widespread literacy in technology and engineering: 


• First, focus on state standards. States should include 
technology and engineering literacy in states' academic 
standards for what students should know and be able to 
do in science. At least one third of U.S. states have already 
adopted standards that explicitly address technology and 
engineering literacy as TEL defines it, and others would do 
well to follow suit, either by adapting other states' stan- 
dards or creating their own . 17 Standards are among the 
most critical tools for encouraging broad access to oppor- 
tunity, because they aim to affect all students in public 
schools, regardless of race, income, or gender. 

• Improve state assessments. By themselves, standards 
are not enough to ensure that technology and engi- 
neering literacy become part of students' experience in 
school. States must create or adopt tests that measure 
students' mastery of those standards, or schools may 
have little incentive to heed the standards. Armed with 
new information on their children's performance on such 
tests, parents could become more effective advocates 
for better hands-on education opportunities in their 
children's schools. 

Most traditional state tests are still ill-suited to the kinds 
of open-ended tasks TEL emphasizes, despite recent 
advances in assessment technology. As a sophisticated 
computer-based test that uses real-world simulations, 
the TEL assessment itself could be one model for a new 
generation of state science tests. 

• Provide better curriculum and teaching materials. 
Teachers need strong curricula and teaching materials 
in technology and engineering to meet new standards. 
Fortunately, such curricula and materials already exist. 
STEMworks, Change the Equation's honor roll of rigor- 
ously-vetted STEM education programs and resources, 
includes powerful curriculum and teaching materials 
that feature hands-on, real-world, open-ended tasks. 
Curricula and programs like Engaging Youth through 
Engineering, Engineering is Elementary, Engineering the 
Future, and Project Lead the Way have already proven 
their impact, and they carefully align themselves to 
states' content standards. 
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These curricula excel in developing skills central to 
TEL's definition of technology and engineering literacy. 
Trained reviewers have carefully evaluated the extent to 
which program curricula encourage students to define 
problems, develop and use models, plan and carry out 
investigations, analyze and interpret data, and design 
solutions. 18 

School district leaders or teachers who would like 
to review curricula not in STEMworks can also turn to 
the EQuIP Rubric 19 for science lessons and units, which 
includes criteria for evaluating how well these materials 
incorporate engineering literacy. 

• Offer teacher professional development in technology 
and engineering literacy. Teachers also need strong pro- 
fessional development to teach such curricula effectively. 
Organizations that provide excellent K-12 engineering and 
technology curricula, like those featured in STEMworks, 
typically also provide professional development to help 
teachers promote the kinds of hands-on, project-based 
learning that are most likely to foster technology and 
engineering literacy. States should invest in professional 
development that promotes that sort of instruction while 
meeting high standards for professional learning. 20 

• Give young people access to better facilities and mate- 
rials, both in and out of schools. All young people need 
places where they can build, take apart, examine, test, 
fix, and invent things. These places do not necessarily 
need to have all the latest equipment, but they do need 
to offer materials and equipment that few young people 
have at home. 

Schools do not have to tackle this challenge alone. 
State and school district leaders should collaborate with 
partners such as business leaders who can donate used 
equipment or community leaders who can help transform 
public spaces in libraries or community centers into public 
workshops and makerspaces. 

• Create out-of-school learning opportunities for more 
young people. State governments, private and corporate 
foundations, community groups, and others should work 
together in an all-hands-on-deck effort to expand access 
to such opportunities in the nation's most vulnerable com- 
munities. Just as important, they should ensure that public 
and private dollars go to programs and strategies that 
are most likely to make a difference. Again, Change the 
Equation's STEMworks honor roll is a valuable source of 
independently-vetted STEM programs that have proven 
their effectiveness. 


• Take advantage of federal resources. The federal Every 
Student Succeeds Act (ESSA) provides resources to 
help states and local communities carry out many of 
the strategies above. ESSA allows states to use federal 
funding to integrate technology and engineering literacy 
into their state tests. U.S. Education Department officials 
encourage state and district education agencies to use 
federal funds from Title II of ESSA for professional devel- 
opment "to support educators as they implement new 
courses, such as ... engineering.'' 21 The Department also 
explicitly allows states and school districts to use federal 
funds to "increase student access to materials and 
equipment needed to support inquiry-based pedagogy 
and active learning," which could enrich makerspaces 
and similar facilities. 22 Finally, ESSA makes federal 
resources available for out-of-school activities through 
the 21st-Century Community Learning Center initiative, 
which in recent years has funneled more than $1 billion 
annually to out-of-school programs that reach children in 
struggling communities. 

Education Department officials and federal legislators 
have taken pains to make funding available to leaders 
who embrace the vision of education embodied in TEL, 
but they do not require states to use the funds this way. It 
is up to STEM education advocates to keep the vision of 
technology and engineering literacy front and center in 
states' education strategies. 

An agenda for shared prosperity 
For years, employers have been alarmed by the looming 
scarcity of computer scientists and engineers, but the TEL 
results suggest that the talent shortage is in fact even 
more far-reaching than that. Young people who are illit- 
erate in technology and engineering will probably never 
become computer professionals or engineers, but they 
will also lack something more fundamental: the ability to 
master many of the challenges that will confront them in 
their lives, their jobs, and their communities. 

As technology continues to co-opt the most routine 
physical and intellectual jobs, more of the remaining jobs 
will require people who can diagnose problems, examine 
them from different perspectives, test possible solutions, and 
figure out what works best. To develop these skills, more 
young people need frequent opportunities to tinker and 
troubleshoot, to design, dismantle, and create. Fostering 
young people's technology and engineering literacy is a criti- 
cal strategy for carrying the nation's founding ideals of equal 
opportunity and shared prosperity well into the 21st century. 
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